{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MultiIndex\n",
    "MultiIndex，即具有多个层次的索引，有些类似于根据索引进行分组的形式。通过多层次索引，我们就可以使用高层次的索引，来操作整个索引组的数据。  \n",
    "## 创建方式\n",
    "### 第一种\n",
    "我们在创建Series或DataFrame时，可以通过给index（columns）参数传递多维数组，进而构建多维索引。【数组中每个维度对应位置的元素，组成每个索引值】  \n",
    "多维索引的也可以设置名称（names属性），属性的值为一维数组，元素的个数需要与索引的层数相同（每层索引都需要具有一个名称）。\n",
    "### 第二种\n",
    "我们可以通过MultiIndex类的相关方法，预先创建一个MultiIndex对象，然后作为Series与DataFrame中的index（或columns）参数值。同时，可以通过names参数指定多层索引的名称。  \n",
    "* from_arrays：接收一个多维数组参数，高维指定高层索引，低维指定底层索引。\n",
    "* from_tuples：接收一个元组的列表，每个元组指定每个索引（高维索引，低维索引）。\n",
    "* from_product：接收一个可迭代对象的列表，根据多个可迭代对象元素的笛卡尔积进行创建索引。\n",
    "\n",
    "from_product相对于前两个方法而言，实现相对简单，但是，也存在局限。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Series创建多层索引， 通过index指定一个多维的列表（数组）。\n",
    "# 单层索引，指定一个一维数组。\n",
    "# s = pd.Series([1, 2, 3, 4], index=[\"a\", \"b\", \"c\", \"d\"])\n",
    "# 多层索引，指定一个多维数组。多维数组中，逐级给出每层索引的值。\n",
    "s = pd.Series([1, 2, 3, 4], index=[[\"A\", \"A\", \"B\", \"B\"], [\"a\", \"b\", \"c\", \"d\"]])\n",
    "# 多于多层索引，每一层都具有一个名字。\n",
    "# s.index.names = [\"abc\", \"def\"]\n",
    "# display(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"2\" halign=\"left\">A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>b</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   A     B\n",
       "   a  b  b\n",
       "0  0  1  2\n",
       "1  3  4  5\n",
       "2  6  7  8"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   a  b\n",
       "0  0  1\n",
       "1  3  4\n",
       "2  6  7"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "0    0\n",
       "1    3\n",
       "2    6\n",
       "Name: (A, a), dtype: int32"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# df = pd.DataFrame([[1, 2, 3], [4, 5, 6], [7, 8, 9]], index=[[\"A\", \"A\", \"B\"], [\"a\", 'b', 'b']])\n",
    "# 很意外，此种方式不成功，我们需要使用numpy提供的数组生成函数才行。\n",
    "# df = pd.DataFrame([[1, 2, 3], [4, 5, 6], [7, 8, 9]], columns=[[\"A\", \"A\", \"B\"], [\"a\", 'b', 'b']])\n",
    "df = pd.DataFrame(np.arange(9).reshape(3, 3), columns=[[\"A\", \"A\", \"B\"], [\"a\", 'b', 'b']])\n",
    "display(df)\n",
    "display(df[\"A\"])\n",
    "display(df[\"A\", \"a\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">A</th>\n",
       "      <th>a</th>\n",
       "      <td>0.954184</td>\n",
       "      <td>0.052612</td>\n",
       "      <td>0.943119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>0.859722</td>\n",
       "      <td>0.267176</td>\n",
       "      <td>0.897029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">B</th>\n",
       "      <th>a</th>\n",
       "      <td>0.049726</td>\n",
       "      <td>0.901836</td>\n",
       "      <td>0.871435</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>0.769453</td>\n",
       "      <td>0.475607</td>\n",
       "      <td>0.350911</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            0         1         2\n",
       "A a  0.954184  0.052612  0.943119\n",
       "  b  0.859722  0.267176  0.897029\n",
       "B a  0.049726  0.901836  0.871435\n",
       "  b  0.769453  0.475607  0.350911"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 通过MultiIndex类的方法进行创建。\n",
    "\n",
    "# 通过列表的方式进行创建。（每个内嵌列表元素指定层次的索引，[[第0层索引], [第1层索引]，……[第n层索引]]）\n",
    "m = pd.MultiIndex.from_arrays([[\"A\", \"A\", \"B\"], [\"a\", \"b\", \"b\"]])\n",
    "\n",
    "# 通过元组构成列表的方式进行创建。[(高层，底层), (高层， 底层), ……]\n",
    "m2 = pd.MultiIndex.from_tuples([(\"A\", \"a\"), (\"A\", \"b\"), (\"B\", \"b\")])\n",
    "df = pd.DataFrame(np.random.random((3, 3)), index=m2)\n",
    "\n",
    "# 通过乘积（笛卡尔积）的方式进行创建。\n",
    "m3 = pd.MultiIndex.from_product([[\"A\", \"B\"], [\"a\", \"b\"]])\n",
    "df = pd.DataFrame(np.random.random((4, 3)), index=m3)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 多层索引操作\n",
    "对于多层索引，同样也支持单层索引的相关操作，例如，索引元素，切片，索引数组选择元素等。我们也可以根据多级索引，按层次逐级选择元素。  \n",
    "多层索引的优势：通过创建多层索引，我们就可以使用高层次的索引，来操作整个索引组的数据。  \n",
    "格式：  \n",
    "* s[操作]\n",
    "* s.loc[操作]\n",
    "* s.iloc[操作]\n",
    "\n",
    "其中，操作可以是索引，切片，数组索引，布尔索引。  \n",
    "\n",
    "### Series多层索引\n",
    "* 通过loc（标签索引）操作，可以通过多层索引，获取该索引所对应的一组值。\n",
    "* 通过iloc（位置索引）操作，会获取对应位置的元素值（与是否多层索引无关）。\n",
    "* 通过s[操作]的行为有些诡异，建议不用。\n",
    " + 对于索引（单级），首先按照标签选择，如果标签不存在，则按照位置选择。\n",
    " + 对于多级索引，则按照标签进行选择。\n",
    " + 对于切片，如果提供的是整数，则按照位置选择，否则按照标签选择。\n",
    " + 对于数组索引, 如果数组元素都是整数，则根据位置进行索引，否则，根据标签进行索引。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A  a    1\n",
       "   b    2\n",
       "B  a    3\n",
       "   b    4\n",
       "dtype: int64"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "A  a    1\n",
       "   b    2\n",
       "B  a    3\n",
       "   b    4\n",
       "dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = pd.Series([1, 2, 3, 4], index=[[\"A\", \"A\", \"B\", \"B\"], [\"a\", \"b\", \"a\", \"b\"]])\n",
    "display(s)\n",
    "# 通过访问外层索引，可以获取该外层索引对应的一组值。\n",
    "# display(s[\"A\"])\n",
    "# 多层索引，不直接直接使用内层索引访问。\n",
    "#  错误\n",
    "# display(s[\"a\"])\n",
    "# 可以通过索引逐层进行访问\n",
    "# display(s[\"A\", \"a\"])\n",
    "\n",
    "# 对于iloc，就是根据位置进行访问，与是否存在多层索引，关系不大。\n",
    "# s.iloc[1]\n",
    "\n",
    "s[[\"A\", \"B\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DataFrame多层索引\n",
    "* 通过loc（标签索引）操作，可以通过多层索引，获取该索引所对应的一组值。\n",
    "* 通过iloc（位置索引）操作，会获取对应位置的一行（与是否多层索引无关）。\n",
    "* 通过s[操作]的行为有些诡异，建议不用。\n",
    " + 对于索引，根据标签获取相应的列（如果是多层索引，则可以获得多列）。\n",
    " + 对于数组索引, 根据标签，获取相应的列（如果是多层索引，则可以获得多列）。\n",
    " + 对于切片，首先按照标签进行索引，然后再按照位置进行索引（取行）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.284917\n",
       "1    0.746716\n",
       "2    0.575233\n",
       "Name: (A, a), dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(np.random.rand(3, 3), index=[[\"A\", \"A\", \"B\"], [\"a\", \"b\", \"b\"]])\n",
    "# df.loc[\"A\", \"a\"]\n",
    "df.iloc[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 交换索引\n",
    "我们可以调用DataFrame对象的swaplevel方法来交换两个层级索引。该方法默认对倒数第2层与倒数第1层进行交换。我们也可以指定交换的层。层次从0开始，由外向内递增（或者由上到下递增），也可以指定负值，负值表示倒数第n层。除此之外，我们也可以使用层次索引的名称来进行交换。  \n",
    "\n",
    "## 索引排序\n",
    "我们可以使用sort_index方法对索引进行排序处理。\n",
    "* level：指定根据哪一层进行排序，默认为最外（上）层。该值可以是数值，索引名，或者是由二者构成的列表。\n",
    "* inplace：是否就地修改。默认为False。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>layer1</th>\n",
       "      <th>layer2</th>\n",
       "      <th>layer3</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">A</th>\n",
       "      <th rowspan=\"2\" valign=\"top\">a1</th>\n",
       "      <th>a2</th>\n",
       "      <td>0.984698</td>\n",
       "      <td>0.615676</td>\n",
       "      <td>0.753877</td>\n",
       "      <td>0.932709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b2</th>\n",
       "      <td>0.493305</td>\n",
       "      <td>0.262877</td>\n",
       "      <td>0.542709</td>\n",
       "      <td>0.687143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">B</th>\n",
       "      <th>b1</th>\n",
       "      <th>c2</th>\n",
       "      <td>0.244724</td>\n",
       "      <td>0.528395</td>\n",
       "      <td>0.653433</td>\n",
       "      <td>0.178914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c1</th>\n",
       "      <th>c2</th>\n",
       "      <td>0.316843</td>\n",
       "      <td>0.366835</td>\n",
       "      <td>0.612388</td>\n",
       "      <td>0.804049</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             0         1         2         3\n",
       "layer1 layer2 layer3                                        \n",
       "A      a1     a2      0.984698  0.615676  0.753877  0.932709\n",
       "              b2      0.493305  0.262877  0.542709  0.687143\n",
       "B      b1     c2      0.244724  0.528395  0.653433  0.178914\n",
       "       c1     c2      0.316843  0.366835  0.612388  0.804049"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>layer3</th>\n",
       "      <th>layer2</th>\n",
       "      <th>layer1</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a2</th>\n",
       "      <th>a1</th>\n",
       "      <th>A</th>\n",
       "      <td>0.984698</td>\n",
       "      <td>0.615676</td>\n",
       "      <td>0.753877</td>\n",
       "      <td>0.932709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b2</th>\n",
       "      <th>a1</th>\n",
       "      <th>A</th>\n",
       "      <td>0.493305</td>\n",
       "      <td>0.262877</td>\n",
       "      <td>0.542709</td>\n",
       "      <td>0.687143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">c2</th>\n",
       "      <th>b1</th>\n",
       "      <th>B</th>\n",
       "      <td>0.244724</td>\n",
       "      <td>0.528395</td>\n",
       "      <td>0.653433</td>\n",
       "      <td>0.178914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c1</th>\n",
       "      <th>B</th>\n",
       "      <td>0.316843</td>\n",
       "      <td>0.366835</td>\n",
       "      <td>0.612388</td>\n",
       "      <td>0.804049</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             0         1         2         3\n",
       "layer3 layer2 layer1                                        \n",
       "a2     a1     A       0.984698  0.615676  0.753877  0.932709\n",
       "b2     a1     A       0.493305  0.262877  0.542709  0.687143\n",
       "c2     b1     B       0.244724  0.528395  0.653433  0.178914\n",
       "       c1     B       0.316843  0.366835  0.612388  0.804049"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(np.random.rand(4, 4), index=[[\"A\", \"A\", \"B\", \"B\"], [\"a1\", \"a1\", \"b1\", \"c1\"], [\"a2\", \"b2\", \"c2\", \"c2\"]])\n",
    "df.index.names = [\"layer1\", \"layer2\", \"layer3\"]\n",
    "display(df)\n",
    "# 多层索引，编号从外向内，0， 1， 2， 3.同时，索引编号也支持负值。\n",
    "# 负值表示从内向外，-1， -2， -3. -1表示最内容。\n",
    "\n",
    "# 交换多层索引的两层。默认交换最内层与倒数第二层。（最里面的两层）\n",
    "# df.swaplevel(0, 2)\n",
    "# 交换多层索引时，我们除了可以指定层次的编号外，也可以指定索引层次的名称。\n",
    "df.swaplevel(\"layer1\", \"layer3\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>layer1</th>\n",
       "      <th>layer2</th>\n",
       "      <th>layer3</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>A</th>\n",
       "      <th>b</th>\n",
       "      <th>b2</th>\n",
       "      <td>0.648595</td>\n",
       "      <td>0.009719</td>\n",
       "      <td>0.251170</td>\n",
       "      <td>0.935658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">B</th>\n",
       "      <th>b</th>\n",
       "      <th>c2</th>\n",
       "      <td>0.038701</td>\n",
       "      <td>0.628436</td>\n",
       "      <td>0.313889</td>\n",
       "      <td>0.085547</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <th>a2</th>\n",
       "      <td>0.257360</td>\n",
       "      <td>0.822571</td>\n",
       "      <td>0.070936</td>\n",
       "      <td>0.719467</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>A</th>\n",
       "      <th>c</th>\n",
       "      <th>c2</th>\n",
       "      <td>0.817126</td>\n",
       "      <td>0.693073</td>\n",
       "      <td>0.309432</td>\n",
       "      <td>0.837334</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             0         1         2         3\n",
       "layer1 layer2 layer3                                        \n",
       "A      b      b2      0.648595  0.009719  0.251170  0.935658\n",
       "B      b      c2      0.038701  0.628436  0.313889  0.085547\n",
       "       a      a2      0.257360  0.822571  0.070936  0.719467\n",
       "A      c      c2      0.817126  0.693073  0.309432  0.837334"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>layer1</th>\n",
       "      <th>layer2</th>\n",
       "      <th>layer3</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">A</th>\n",
       "      <th>b</th>\n",
       "      <th>b2</th>\n",
       "      <td>0.648595</td>\n",
       "      <td>0.009719</td>\n",
       "      <td>0.251170</td>\n",
       "      <td>0.935658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>c</th>\n",
       "      <th>c2</th>\n",
       "      <td>0.817126</td>\n",
       "      <td>0.693073</td>\n",
       "      <td>0.309432</td>\n",
       "      <td>0.837334</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">B</th>\n",
       "      <th>a</th>\n",
       "      <th>a2</th>\n",
       "      <td>0.257360</td>\n",
       "      <td>0.822571</td>\n",
       "      <td>0.070936</td>\n",
       "      <td>0.719467</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <th>c2</th>\n",
       "      <td>0.038701</td>\n",
       "      <td>0.628436</td>\n",
       "      <td>0.313889</td>\n",
       "      <td>0.085547</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             0         1         2         3\n",
       "layer1 layer2 layer3                                        \n",
       "A      b      b2      0.648595  0.009719  0.251170  0.935658\n",
       "       c      c2      0.817126  0.693073  0.309432  0.837334\n",
       "B      a      a2      0.257360  0.822571  0.070936  0.719467\n",
       "       b      c2      0.038701  0.628436  0.313889  0.085547"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(np.random.rand(4, 4), index=[[\"A\", \"B\", \"B\", \"A\"], [\"b\", \"b\", \"a\", \"c\"], [\"b2\", \"c2\", \"a2\", \"c2\"]])\n",
    "df.index.names = [\"layer1\", \"layer2\", \"layer3\"]\n",
    "display(df)\n",
    "# 索引排序，默认进行最外层的排序。\n",
    "# df.sort_index()\n",
    "# 自定义排序的层次。\n",
    "df.sort_index(level=1)\n",
    "# 也可以通过索引的名称进行排序。\n",
    "df.sort_index(level=\"layer1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 索引堆叠\n",
    "通过DataFrame对象的stack方法，可以进行索引堆叠，即将指定层级的列转换成行。\n",
    "level：指定转换的层级，默认为-1。\n",
    "## 取消堆叠\n",
    "通过DataFrame对象的unstack方法，可以取消索引堆叠，即将指定层级的行转换成列。\n",
    "level：指定转换的层级，默认为-1。\n",
    "fill_value：指定填充值。默认为NaN。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>layer1</th>\n",
       "      <th>layer2</th>\n",
       "      <th>layer3</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>A</th>\n",
       "      <th>b</th>\n",
       "      <th>b2</th>\n",
       "      <td>0.630500</td>\n",
       "      <td>0.974254</td>\n",
       "      <td>0.963365</td>\n",
       "      <td>0.618260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">B</th>\n",
       "      <th>b</th>\n",
       "      <th>c2</th>\n",
       "      <td>0.547420</td>\n",
       "      <td>0.756999</td>\n",
       "      <td>0.149537</td>\n",
       "      <td>0.756407</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <th>a2</th>\n",
       "      <td>0.115701</td>\n",
       "      <td>0.476125</td>\n",
       "      <td>0.179361</td>\n",
       "      <td>0.585364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>A</th>\n",
       "      <th>c</th>\n",
       "      <th>c2</th>\n",
       "      <td>0.684255</td>\n",
       "      <td>0.874848</td>\n",
       "      <td>0.555394</td>\n",
       "      <td>0.790719</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             0         1         2         3\n",
       "layer1 layer2 layer3                                        \n",
       "A      b      b2      0.630500  0.974254  0.963365  0.618260\n",
       "B      b      c2      0.547420  0.756999  0.149537  0.756407\n",
       "       a      a2      0.115701  0.476125  0.179361  0.585364\n",
       "A      c      c2      0.684255  0.874848  0.555394  0.790719"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "layer1  layer2  layer3   \n",
       "A       b       b2      0    0.630500\n",
       "                        1    0.974254\n",
       "                        2    0.963365\n",
       "                        3    0.618260\n",
       "B       b       c2      0    0.547420\n",
       "                        1    0.756999\n",
       "                        2    0.149537\n",
       "                        3    0.756407\n",
       "        a       a2      0    0.115701\n",
       "                        1    0.476125\n",
       "                        2    0.179361\n",
       "                        3    0.585364\n",
       "A       c       c2      0    0.684255\n",
       "                        1    0.874848\n",
       "                        2    0.555394\n",
       "                        3    0.790719\n",
       "dtype: float64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(np.random.rand(4, 4), index=[[\"A\", \"B\", \"B\", \"A\"], [\"b\", \"b\", \"a\", \"c\"], [\"b2\", \"c2\", \"a2\", \"c2\"]])\n",
    "df.index.names = [\"layer1\", \"layer2\", \"layer3\"]\n",
    "display(df)\n",
    "# 取消堆叠，如果没有匹配的数据，则显示空值NaN。\n",
    "# df.unstack()\n",
    "# 我们可以指定值去填充NaN（空值）。\n",
    "# df.unstack(fill_value=50000)\n",
    "# unstack默认会将最内层取消堆叠，我们也可以自行来指定层次。\n",
    "# df.unstack(0)\n",
    "# 除了指定层次序号外，我们也可以指定层次（索引）的名称。\n",
    "# df.unstack(\"layer1\")\n",
    "\n",
    "# 进行堆叠  列-》行      取消堆叠   行-》列\n",
    "df.stack()\n",
    "# stack堆叠也可以指定层次。\n",
    "df.stack(0)\n",
    "# stack堆叠也可以通过索引名进行操作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 设置索引\n",
    "在DataFrame中，如果我们需要将现有的某一（几）列作为索引列，可以调用set_index方法来实现。\n",
    "* drop：是否丢弃作为新索引的列，默认为True。\n",
    "* append：是否以追加的方式设置索引，默认为False。\n",
    "* inplace：是否就地修改，默认为False。\n",
    "\n",
    "## 重置索引\n",
    "调用在DataFrame对象的reset_index，可以重置索引。该操作与set_index正好相反。\n",
    "* level：重置索引的层级，默认重置所有层级的索引。如果重置所有索引，将会创建默认整数序列索引。\n",
    "* drop：是否丢弃重置的索引列，默认为False。\n",
    "* inplace：是否就地修改，默认为False。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>n1</th>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>n2</th>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>n3</th>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>n4</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      age\n",
       "name     \n",
       "n1     15\n",
       "n2     20\n",
       "n3     17\n",
       "n4      8"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame({\"pk\":[1, 2, 3, 4], \"name\":[\"n1\", \"n2\", \"n3\", \"n4\"], \"age\":[15, 20, 17, 8]})\n",
    "# 字典是无序的，如果需要顺序调整。\n",
    "# df = df[[\"pk\", \"name\", \"age\"]]\n",
    "# 也可以通过reindex来重新组织索引。\n",
    "# df.reindex([\"pk\", \"name\", \"age\"], axis=1)\n",
    "\n",
    "# 设置参数指定的列，充当新的索引。\n",
    "# df.set_index(\"pk\")\n",
    "# 也可以设置层级索引。\n",
    "# df.set_index([\"pk\", \"name\"])\n",
    "\n",
    "# 默认情况下，充当新索引的列会被丢弃，我们可以通过drop=False设置依然保留。\n",
    "# df = df.set_index(\"pk\", drop=False)\n",
    "# append用来设置是否以追加的形式设置索引（层级索引）。默认为False（取代以前的索引）。\n",
    "# df.set_index(\"name\", append=True)\n",
    "\n",
    "\n",
    "# df.set_index(\"pk\", inplace=True)\n",
    "# 重置索引。默认情况下重置所有层次的索引。如果所有层次的索引均被重置，则重新生成整数序列的索引。\n",
    "# df.reset_index()\n",
    "\n",
    "df.set_index([\"pk\", \"name\"], inplace=True)\n",
    "# 我们也可以指定重置索引的层次。【如果重置索引之后，还有索引层次（没有重置所有层次的索引），则不会重新生成整数序列的索引。】\n",
    "# df.reset_index(0, inplace=True)\n",
    "\n",
    "# 重置索引后，默认重置索引会充当新的列，如果不需要重置的索引充当新的列，可以指定drop=True。\n",
    "df.reset_index(0, drop=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 分组与聚合\n",
    "分组与聚合操作与数据库中的分组与聚合相似。  \n",
    "## groupby分组\n",
    "我们可以通过groupby方法来对Series或DataFrame对象实现分组操作。该方法会返回一个分组对象。不过，如果直接查看（输出）该对象，并不能看到任何的分组信息。\n",
    "* groups（属性）：返回一个字典类型对象，包含分组信息。\n",
    "* size：返回每组记录的数量。\n",
    "* discribe：分组查看统计信息。\n",
    "\n",
    "## 迭代\n",
    "我们也可以使用for循环来对分组对象进行迭代。迭代每次会返回一个元组，第1个元素为用来分组的key，第2个元素为该组对应的数据。\n",
    "\n",
    "## 分组的方式\n",
    "使用groupby进行分组时，分组的参数可以是如下的形式：\n",
    "* 索引名：根据该索引进行分组。\n",
    "* 索引名构成的数组：根据数组中每个索引进行分组。\n",
    "* 字典或Series：key指定索引，value指定分组依据，即value值相等的，会分为一组。\n",
    "* 函数：接受索引，返回分组依据的value值。\n",
    "\n",
    "## apply\n",
    "对分组对象，可以调用apply函数，该函数接收每个组的数据，返回操作之后的结果。apply最后会将每个组的操作结果进行合并（concat）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame({\"部门\":[\"A\", \"A\", \"B\", \"B\"], \"利润\":[10, 20, 15, 28], \"人员\":[\"a\", \"b\", \"c\", \"d\"], \"年龄\":[20, 15, 18, 30]})\n",
    "# 根据部门进行分组，返回一个分组对象。\n",
    "# display(df)\n",
    "g = df.groupby(\"部门\")\n",
    "# 返回分组对象的信息。\n",
    "# display(g.groups)\n",
    "# 返回每组的数量。\n",
    "# display(g.size())\n",
    "# 返回每组的统计信息。\n",
    "# display(g.describe())\n",
    "\n",
    "# 分组信息不像列表那样，我们可以直接输出查看。（类似于迭代器的机制，是需要时，动态进行计算的。）\n",
    "# 我们可以通过for循环来查看分组对象的数据。类似字典的形式，key-> 分组列的值。 value-》分组对应的数据记录。\n",
    "# for k, v in g:\n",
    "#     display(k, v)\n",
    "    \n",
    "# display(df)\n",
    "# 在分组上进行聚合统计。\n",
    "\n",
    "# 在整个DataFrame上进行聚合。\n",
    "# g.sum()\n",
    "# 在指定的列上进行聚合。\n",
    "# g[\"利润\"].sum()\n",
    "\n",
    "# display(g)\n",
    "# display(g[\"利润\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame({\"部门\":[\"A\", \"A\", \"B\", \"B\"],\"小组\":[\"g1\", \"g2\", \"g1\", \"g2\"], \"利润\":[10, 20, 15, 28], \"人员\":[\"a\", \"b\", \"c\", \"d\"], \"年龄\":[20, 15, 18, 30]})\n",
    "display(df)\n",
    "# 分组的方式\n",
    "# 1 进行单一的分组，提供单一的标签。\n",
    "# df.groupby(\"部门\")\n",
    "# 2 进行多分组，提供多标签构成的列表。\n",
    "# g = df.groupby([\"部门\", \"小组\"])\n",
    "\n",
    "# display(g)\n",
    "# i = g.__iter__()\n",
    "# i = iter(g)\n",
    "# i.__next__()\n",
    "# next(i)\n",
    "# 3 提供一个字典(Series)，key提供索引， value提供组名。结果就会根据value相同的记录，分到一组。\n",
    "# g = df.groupby({0:1, 1:1, 2:2, 3:2})\n",
    "# 我们还可以根据列进行分组。将axis参数指定为1。\n",
    "# g = df.groupby({\"人员\":1, \"利润\":1, \"小组\":2, \"年龄\":2, \"部门\":2}, axis=1)\n",
    "\n",
    "# 4 提供一个函数。函数需要具有一个参数。用来依次接收索引值。函数还需要具有一个返回值，用来指定组。\n",
    "# 如果看成字典的形式，函数接收key，返回value。\n",
    "g = df.groupby(lambda x: x % 2)\n",
    "\n",
    "# for k, v in g:\n",
    "#     display(k, v)\n",
    "# display(g.sum())\n",
    "\n",
    "# DataFrame 进行聚合时,默认会包含所有列，我们可以指定numeric_only参数为True，这样只会聚合（统计）数值类型的行（列）。\n",
    "# display(df.sum(numeric_only=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 分组的方式：\n",
    "# 每个索引，会依次传递到函数中（index）。\n",
    "def group_helper(index):\n",
    "#     if index > 5:\n",
    "#         return 0\n",
    "#     else:\n",
    "#         return 1\n",
    "    return 0 if index > 2 else 1\n",
    "# 对于传递进行的索引 0， 1， 2， 3\n",
    "# 每个索引对应的返回值\n",
    "# 0 -> 1\n",
    "# 1 -> 1\n",
    "# 2 -> 1\n",
    "# 3 -> 0\n",
    "# 根据函数返回的value值，索引0， 1, 2 分到一组，索引3分到一组。\n",
    "\n",
    "# 类似的，对于Series或字典来说，\n",
    "# df.groupby({0:1, 1:1, 2:1, 3:0})\n",
    "    \n",
    "# df.groupby(group_helper)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 分组对象也可以进行apply。apply接收一个函数（参数）。函数具有一个参数，用来依次接收每一个分组数据，并且返回每一个分组数据的结果。\n",
    "df = pd.DataFrame({\"部门\":[\"A\", \"A\", \"B\", \"B\"],\"小组\":[\"g1\", \"g2\", \"g1\", \"g2\"], \"利润\":[10, 20, 15, 28], \n",
    "                   \"人员\":[\"a\", \"b\", \"c\", \"d\"], \"年龄\":[20, 15, 18, 30]})\n",
    "display(df)\n",
    "g = df.groupby(lambda x: x % 2)\n",
    "# apply当前实现的细节：对于第一个分组，会调用函数两次。（但是，不影响apply最后进行concat合并的结果）\n",
    "# g.apply(lambda x: x.sum(numeric_only=True))\n",
    "# g.apply(lambda x: x.max(numeric_only=True) - x.min(numeric_only=True))\n",
    "g.apply(lambda x: x.describe().T)\n",
    "\n",
    "# display(g.describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 聚合\n",
    "可以在分组对象上进行聚合（多个值变成一个值）。例如，mean()，sum()等。  \n",
    "除此之外，我们也可以调用agg方法，实现自定义的聚合方式。函数接收一行（一列），返回该行（列）聚合后的结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame({\"利润\":[10, 20, 15, 28], \"年龄\":[20, 15, 18, 30]})\n",
    "display(df)\n",
    "# DataFrame与分组对象可以调用agg / aggregate方法，来实现聚合操作。\n",
    "\n",
    "# 关于agg方法，可以接收的参数类型：\n",
    "# 1 函数说明字符串，例如，sum，mean，std\n",
    "# display(df.mean())\n",
    "# display(df.agg(\"mean\"))\n",
    "# 2 多个函数字符串构成的列表\n",
    "# display(df.agg([\"mean\", \"sum\", \"max\"]))\n",
    "# 3 字典 key：指定列标签，value：指定统计的信息。\n",
    "# 优势：可以针对不同的列， 提供不同的聚合信息。\n",
    "# display(df.agg({\"利润\":[\"mean\", \"sum\"] ,  \"年龄\":[\"max\", \"min\"]}))\n",
    "# 4 函数, 可以实现自己的聚合方式。函数具有一个参数，用来接收DataFrame传递过来的每一列（行），返回操作之后的结果。\n",
    "# df.agg(lambda x: x.mean())\n",
    "\n",
    "# 分组对象的agg，与DataFrame相似。\n",
    "g = df.groupby(lambda x: x % 2)\n",
    "# g.mean()\n",
    "# g.agg(\"mean\")\n",
    "# g.agg([\"mean\", \"sum\"])\n",
    "# display(g.agg({\"利润\":[\"mean\", \"sum\"] ,  \"年龄\":[\"max\", \"min\"]}))\n",
    "# 函数接收的是每个组的每一个列（行）。函数的返回值作为最终聚合的结果。\n",
    "g.agg(lambda x: x.max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 练习\n",
    "按照天来统计电影信息的票房信息。（票房总数，平均值，最大值，最小值，电影数量）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"spider.csv\", header=None)\n",
    "# 首先，进行数据的过滤，=》 只保留电影的信息记录\n",
    "t = df[df[1].str.startswith(\"http://www.movie.com/bor/\")]\n",
    "# 3, 4两列对我们没有任何帮助，可以删除。\n",
    "t.drop([3, 4], axis=1, inplace=True)\n",
    "# 查看电影信息是否存在空值。是否存在重复的数据，删除。\n",
    "# 通过info查看，没有空值。\n",
    "# t.info()\n",
    "# 查看重复的记录。\n",
    "# t[t.duplicated()]\n",
    "# 删除重复的记录。\n",
    "t.drop_duplicates(inplace=True)\n",
    "# 针对第2列，拆分,并扩展。\n",
    "sp = t[2].str.split(\";\", expand=True)\n",
    "# 删除没有用的8,9列。\n",
    "sp.drop([8, 9], axis=1, inplace=True)\n",
    "# 对于第7列（票房）列进行调整。提取其中的数值信息。注意：需要进行类型转换\n",
    "sp[7] = sp[7].str.replace(\"票房（万）\", \"\").astype(np.float64)\n",
    "# 正则表达式的提取操作。\n",
    "# sp[7] = sp[7].str.extract(r\"([0-9.]+)\")\n",
    "# 与原有的DataFrame合并。[concat  merge   join]\n",
    "result = pd.concat((t, sp), axis=1)\n",
    "# result.columns.size\n",
    "# len(result.columns)\n",
    "# 硬编码，不要使用。\n",
    "# result.columns = list(range(11))\n",
    "result.columns = list(range(result.columns.size))\n",
    "# 进行groupby时，不能含有索引名称相同的列。否则会产生错误。\n",
    "g = result.groupby(0)\n",
    "# g[10].sum().sort_values(ascending=False)\n",
    "g.agg([\"sum\", \"mean\", \"max\", \"min\", \"count\"])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
